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09/493,013 



Examiner 

Hussein Akhavannik 



Appllcant(s) 

KANEKO ET AL 



Art Unit 

2621 



- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this cxsmmunication. 

- If the period for reply spedfied above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is spedfied above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent temn adjustment. See 37 CFR 1.704(b). 

Status 

1 )□ Responsive to communication(s) filed on . 

2a)S This action is FINAL. 2b)n This action is non-final. 

3) 0 Since this application is in condition for allowance except for fornnal nnatters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 
Disposition of Claims 

4) 13 Claim(s) 1-24 is/are pending in the application. 

4a) Of the above claim(s) 1-8 and 15-20 is/are withdrawn from consideration. 

5) 0 Claim(s) is/are allowed. 

6) 13 Claim(s) 9-14 and 21-24 is/are rejected. 
/)□ Claim(s) is/are objected to. 

8) \Z\ Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)K The drawing(s) filed on 18 July 2003 is/are: a)S accepted or b)^ objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
11 )□ The proposed drawing correction filed on is: a)^ approved b)n disapproved by the Examiner. 

If approved, corrected drawings are required In reply to this Office action. 

12) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

1 3) ^ Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 1 9(a)-(d) or (f). 

a)[ElAII b)n Some*c)n None of: 

1 Certified copies of the priority documents have been received. 

2.n Certified copies of the priority documents have been received in Application No. . 



3.n Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) n Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 
Attachment(s) 



1 ) Kl Notice of References Cited (PTO-892) 

2) O Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) □ Information Disclosure Statement(s) (PTO-1449) Paper No(s). 



4) n Interview Summary (PTO-413) Paper No(s). 

5) CD Notice of Infonnal Patent Application (PTO-152) 

6) □ Other: 



U.S. Patent and Trademark Office 
PTOL-326 (Rev. 04-01) 
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DETAILED ACTION 



Election/Restrictions 



1. This application contains claims 1-8 and 15-20 drawn to an invention nonelected with 
traverse in Paper No. 6. A complete reply to the final rejection must include cancellation of 
nonelected claims or other appropriate action (37 CFR 1.144) See MPEP § 821.01. 



2. The amendments to the specification overcome the Examiner's objection cited in 
paragraph 3 of the previous office action. 

Drawings 

3. The drawings were received on July 18, 2003. These drawings are accepted. 

Response to Arguments 

4. Applicant's arguments with respect to claims 9-14 have been considered but are moot in 
view of the new ground(s) of rejection. 



5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 



6. Claims 9-14, 21, and 23 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Palmer (U.S. Patent No. 5,684,715) in view of Itokawa (U.S. Patent No. 6,404,901), and further 
in view of Ito et al (U.S. Patent No. 6,445,409). 
Referring to claim 9, 



Response to Amendment 



Claim Rejections - 35 USC §103 
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i. Extracting feature data of a predetermined object and feature data of a background 
area from a frame of an input video is not explicitly explained by Palmer. Palmer does 
explain extracting feature data of a predetermined object in colimm 6, lines 8-29. The 
attribute generator is responsible for determining characteristics of an object such as 
position, shape, size, and layering information for each video object. However, Palmer 
does not extract feature data of the background area from a frame. Itokawa illustrates 
extracting both the foreground area and the background area from an image input in 
figure 8, reference numbers 102 and 104. Itokawa fiirther explains that the background 
can be extracted whether it moves or does not move in column 9, lines 21-30. The 
extraction of feature data from the background is explained by Itokawa in column 9, lines 
42-50. In order to save spatial and temporal information about an entire frame, it would 
be necessary to extract both the object and the background information from a frame. 
Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to extract feature data of a predetermined object and feature data of 
the background area from a frame. 

ii. Describing the feature data of the predetermined object and the feature data of the 
background area as a descriptor of the frame, the feature data of the predetermined object 
including color information of an area of the predetermined object is not explicitly 
explained by Palmer or Itokawa. Palmer does illustrate describing the feature data of a 
predetermined object as a video descriptor in figure 3. Furthermore, Itokawa explains 
determining the feature data of a background area of a frame corresponding to part i of 
this claim. However, the system of Palmer and Itokawa does not include color 
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information as feature data of a predetermined object. Ito et al explain including color 
distribution in order to track and monitor an object in column 10, line 57 to column 1 1, 
line 3. By determining the color distribution of an object and storing it in the object 
descriptor of Palmer and Itokawa, an object is better defined and able to be tracked more 
accurately throughout a series of frames. Therefore, it would have been obvious to one 
of ordinary skill in the art at the time the invention was made to describe the feature data 
of the predetermined object, including color information of an area of the predetermined 
object, and the feature data of the background area as a descriptor of the fi'ame so that 
complete feature data about an entire firame of video is recorded and object tracking is 
performed more accurately. 

iii. Attaching the descriptor to the frame is illustrated by Palmer in figure 3, as the 

video object descriptor 60 corresponds to object 64 of firame 51. 

Referring to claim 10, describing a difference between the feature data of the 
predetermined object and the feature data of the background area as the descriptor is not 
explicitly explained by Palmer. However, Itokawa explains in column 9, lines 6-30 that the 
background data and the foreground data are separated, thereby describing the difference 
between the feature data and the background data. Itokawa fiirther illustrates in figure 26, 
illustrates the difference between the background motion vectors (601) and the object motion 
vectors. In order to separate a predetermined object fi-om the background area of a frame, it 
would be necessary to determine a difference between a feature of the object and the 
background. Therefore, it would have been obvious to one of ordinary skill in the art at the time 
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the invention was made to describe a difference between the feature data of a predetermined 
object and the feature data of the background area. 
Referring to claim 1 1, 

i. The feature data of the predetermined object including at least position, outward 
form or size, and moving information of the object is explained by Palmer in column 4, 
lines 51-59 and column 5, lines 7-11. The descriptor is explained to include spatial 
information including the position and size of each object and the temporal information 
includes action information that specifies movement of the object. 

ii. The feature data of the background area including at least the moving information 
of the background area is not explicitly explained by Palmer. However, Itokawa explain 
that a background area may be moving and that the background movement maybe be 
calculated as the motion value vector, Vb in column 9, lines 21-30 and illustrated in 
figure 11. In order to save information of an entire firame when the background is 
moving, as suggested by Itokawa, it would be necessary to extract both the moving 
information of the object and the moving information of the background. Therefore, it 
would have been obvious to one of ordinary skill in the art at the time the invention was 
made to include the moving information of the background in the feature data of the 
background. 

Referring to claim 12, the descriptor including: 

i. A frame number is illustrated by Palmer in figure 4, as each fi-ame has its own 
corresponding frame ID. 
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ii. A pointer to a next descriptor is explained by Palmer in column 5, lines 7-11. The 
action information in the descriptors will allow the computer to jump from one frame 
sequence to another frame sequence, which also would contain a descriptor. 

iii. The feature data of the background area corresponds to claim 1 1 . 

iv. The feature data of each object in the frame is explained by Palmer in column 5, 
lines 12-26. Palmer explains that there will be a separate video descriptor for each of the 
objects in the frames. 

Referring to claim 13, the descriptor being created from a corresponding frame at an 
interval of a predetermined number of frames in the input video is explained by Palmer in 
column 6, lines 23-29 and illustrated in figure 4. The interval of frames in the input video is one. 

Referring to claim 14, this claim includes all the limitations of claim 14, but is performed 
on a computer readable medium. Palmer illustrates a computer readable medium capable of 
performing the method of claim 9 in figure 2 as Disk (15). 

Referring to claims 21 and 23, the feature data of the predetermined object including at 
least one of a moving vector and an affine transformation coefficient is illustrated by Palmer in 
figure 3 by the action and the action parameters of the video object descriptor (60), respectively. 
Palmer explains in column 6, lines 63-66 that the object tracking unit creates a history of objects 
and how they move through successive frames, corresponding to the claimed moving vector. 
Palmer then explains that the output from the generator (70 of figure 4, which includes the object 
tracking unit, 72) is stored as a video object descriptor in column 7, lines 5-9. The action 
parameters and size , which are stored by Palmer in the video object descriptor, correspond to the 
claimed affine transformation coefficient. 
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7. Claims 22 and 24 are rejected under 35 U.S.C. 103(a) as being unpatentable over Palmer 
in view of Itokawa, in view of Ito et al, and further in view of Chen (U.S. Patent No. 6,278,466). 
Referring to claims 22 and 24, 

i. The feature data of the backgroimd including at least one of color information is 
not explicitly explained by Palmer or Itokawa. However, Ito et al explain using the color 
distribution of an object in order to track an object, corresponding to claims 1 and 14. 
Color information of a background, as determined by Ito et al, can also be saved in the 
video descriptor of Palmer and Itokawa in order to describe an entire frame of video. 
Furthermore, by saving the color information of an object and a background, the frame is 
more accurately described, leading to more accurate object tracking at the cost of 
computation power. Therefore, it would have been obvious to one of ordinary skill in the 
art at the time the invention was made to include color information as feature data of a 
background in order to track an object through video more accurately. 

ii. The feature data of the background including at least one of camera-work 
information describing an operation of a camera used for the input video is not explicitly 
explained by Palmer or Itokawa or Ito et al. However, Chen explains a text descriptor of 
background frame data including text descriptions of "Camera Still", "Camera Pan", and 
"Camera Zoom" in column 22, line 64 to column 23, line 15. By including camera- work 
information in a video object descriptor, the background motion will be better defined in 
the system of Palmer, Itokawa, and Ito et al It would have been an obvious matter of 
design choice to modify the video object descriptor of Palmer, Itokawa, and Ito et al to 
include camera-work information, since the Applicant has not disclosed that having 
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camera-work information solves any stated problem or is of any particular purpose. 
Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to include cam era- work information in the video object descriptor so 
that the background motion of the video sequence is more accurately described. 



8. THIS ACTION IS MADE FINAL. AppUcant is reminded of the extension of time 
policy as set forth in 37 CFR 1 . 1 36(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS fi-om the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the maiUng date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS fi*om the mailing 
date of this final action. 

9. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 



Hennessey et al (U.S. Patent No. 6,014,461) - To exhibit generating a high level 
descriptor of an object or anomaly including size, shape, color, and sharpness as 
explained in the abstract. 

Golshani et al (U.S. Patent No. 6,594,386) - To exhibit indexing and retrieving 
images by their color content and spatial distribution as illustrated in figure 2. 



Conclusion 
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Zlatsin et al (U.S. Patent No. 6,377,296) - To exhibit classifying an object 
according to attributes including size, color, ... as illustrated in figure 6. 
Jeannin (U.S. Patent No. 6,587,574) - To exhibit using a descriptor in video to 
describe an object, wherein the object ID includes location, color, size, shape etc 
information as explained in colunm 10, lines 50-63 and illustrated in figure 4. 



10. Any inquiry concerning this communication or earlier communications fi-om the 
examiner should be directed to Hussein Akhavannik whose telephone number is (703)306-4049. 
The examiner can normally be reached on M-F 8:30-5:00. 

If attempts to reach the examiner by telephone are unsuccessfiil, the examiner's 
supervisor, Leo H. Boudreau can be reached on (703)305-4706. The fax phone number for the 
organization where this application or proceeding is assigned is (703) 872-9306. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is (703)305-3900. 



Hussein Akhavannik 
October 2, 2003 




